Convolutional Neural Networks¶

Project: Write an Algorithm for Landmark Classification¶

A simple app¶

In this notebook we build a very simple app that uses our exported model.

? Note how we are not importing anything from our source code (we do not use any module from the src directory). This is because the exported model, differently from the model weights, is a standalone serialization of our model and therefore it does not need anything else. You can ship that file to anybody, and as long as they can import torch, they will be able to use your model. This is very important for releasing pytorch models to production.

Test your app¶

Go to a search engine for images (like Google Images) and search for images of some of the landmarks, like the Eiffel Tower, the Golden Gate Bridge, Machu Picchu and so on. Save a few examples locally, then upload them to your app to see how your model behaves!

The app will show the top 5 classes that the model think are most relevant for the picture you have uploaded

In [4]:
!pip install -r requirements.txt | grep -v "already satisfied"

Defaulting to user installation because normal site-packages is not writeable
In [5]:
from src.helpers import setup_env

# If running locally, this will download dataset (make sure you have at 
# least 2 Gb of space on your hard drive)
setup_env()

GPU *NOT* available. Will use CPU (slow)
Dataset already downloaded. If you need to re-download, please delete the directory /data/DLND/C2/landmark_images
Reusing cached mean and std
In [6]:
!jupyter nbextension enable --py widgetsnbextension

Enabling notebook extension jupyter-js-widgets/extension...
      - Validating: OK
In [7]:
from ipywidgets import VBox, Button, FileUpload, Output, Label
from PIL import Image
from IPython.display import display
import io
import numpy as np
import torchvision
import torchvision.transforms as T
import torch

# Decide which model you want to use among the ones exported
learn_inf = torch.jit.load("checkpoints/transfer_exported.pt")# YOUR CODE HERE

def on_click_classify(change):

    # Load image that has been uploaded
    fn = io.BytesIO(btn_upload.data[-1])

    img = Image.open(fn)
    img.load()

    # Let's clear the previous output (if any)
    out_pl.clear_output()

    # Display the image
    with out_pl:

        ratio = img.size[0] / img.size[1]
        c = img.copy()
        c.thumbnail([ratio * 200, 200])
        display(c)

    # Transform to tensor
    timg = T.ToTensor()(img).unsqueeze_(0)

    # Calling the model
    softmax = learn_inf(timg).data.cpu().numpy().squeeze()
    
    # Get the indexes of the classes ordered by softmax
    # (larger first)
    idxs = np.argsort(softmax)[::-1]
    
    # Loop over the classes with the largest softmax
    for i in range(5):
        # Get softmax value
        p = softmax[idxs[i]]
    
        # Get class name
        landmark_name = learn_inf.class_names[idxs[i]]
        
        labels[i].value = f"{landmark_name} (prob: {p:.2f})"


# Putting back btn_upload to a widget for next cell
btn_upload = FileUpload()

btn_run = Button(description="Classify")
btn_run.on_click(on_click_classify)

labels = []
for _ in range(5):
    labels.append(Label())

out_pl = Output()
out_pl.clear_output()

wgs = [Label("Please upload a picture of a landmark"), btn_upload, btn_run, out_pl]
wgs.extend(labels)

VBox(wgs)

SMToutput.png

(optional) Standalone app or web app¶

You can run this notebook as a standalone app on your computer by following these steps:

  1. Download this notebook in a directory on your machine
  2. Download the model export (for example, checkpoints/transfer_exported.pt) in a subdirectory called checkpoints within the directory where you save the app.ipynb notebook
  3. Install voila if you don't have it already (pip install voila)
  4. Run your app: voila app.ipynb --show_tracebacks=True
  5. Customize your notebook to make your app prettier and rerun voila

You can also deploy this app as a website using Binder: https://voila.readthedocs.io/en/stable/deploy.html#deployment-on-binder

Create your submission archive¶

Now that you are done with your project, please run the following cell. It will generate a file containing all the code you have written, as well as the notebooks. Please submit that file to complete your project

In [8]:
!pip install --upgrade nbconvert

Defaulting to user installation because normal site-packages is not writeable
Collecting nbconvert
  Downloading nbconvert-6.5.0-py3-none-any.whl (561 kB)
     |████████████████████████████████| 561 kB 4.9 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: pygments>=2.4.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert) (2.5.2)
Requirement already satisfied, skipping upgrade: pandocfilters>=1.4.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert) (1.4.2)
Collecting nbclient>=0.5.0
  Downloading nbclient-0.6.0-py3-none-any.whl (70 kB)
     |████████████████████████████████| 70 kB 6.4 MB/s  eta 0:00:01
Collecting jupyterlab-pygments
  Downloading jupyterlab_pygments-0.2.2-py2.py3-none-any.whl (21 kB)
Collecting tinycss2
  Downloading tinycss2-1.1.1-py3-none-any.whl (21 kB)
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.1.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting nbformat>=5.1
  Downloading nbformat-5.3.0-py3-none-any.whl (73 kB)
     |████████████████████████████████| 73 kB 2.3 MB/s  eta 0:00:01
Requirement already satisfied, skipping upgrade: entrypoints>=0.2.2 in /opt/conda/lib/python3.7/site-packages (from nbconvert) (0.3)
Collecting jinja2>=3.0
  Downloading Jinja2-3.1.1-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 14.2 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: defusedxml in /opt/conda/lib/python3.7/site-packages (from nbconvert) (0.6.0)
Requirement already satisfied, skipping upgrade: bleach in /opt/conda/lib/python3.7/site-packages (from nbconvert) (3.1.1)
Requirement already satisfied, skipping upgrade: beautifulsoup4 in /opt/conda/lib/python3.7/site-packages (from nbconvert) (4.10.0)
Collecting traitlets>=5.0
  Downloading traitlets-5.1.1-py3-none-any.whl (102 kB)
     |████████████████████████████████| 102 kB 9.7 MB/s eta 0:00:01
Collecting jupyter-core>=4.7
  Downloading jupyter_core-4.10.0-py3-none-any.whl (87 kB)
     |████████████████████████████████| 87 kB 6.4 MB/s  eta 0:00:01
Requirement already satisfied, skipping upgrade: packaging in /opt/conda/lib/python3.7/site-packages (from nbconvert) (20.1)
Requirement already satisfied, skipping upgrade: mistune<2,>=0.8.1 in /opt/conda/lib/python3.7/site-packages (from nbconvert) (0.8.4)
Collecting jupyter-client>=6.1.5
  Downloading jupyter_client-7.2.2-py3-none-any.whl (130 kB)
     |████████████████████████████████| 130 kB 25.2 MB/s eta 0:00:01
Collecting nest-asyncio
  Downloading nest_asyncio-1.5.5-py3-none-any.whl (5.2 kB)
Requirement already satisfied, skipping upgrade: webencodings>=0.4 in /opt/conda/lib/python3.7/site-packages (from tinycss2->nbconvert) (0.5.1)
Collecting fastjsonschema
  Downloading fastjsonschema-2.15.3-py3-none-any.whl (22 kB)
Requirement already satisfied, skipping upgrade: jsonschema>=2.6 in /opt/conda/lib/python3.7/site-packages (from nbformat>=5.1->nbconvert) (3.2.0)
Requirement already satisfied, skipping upgrade: six>=1.9.0 in /opt/conda/lib/python3.7/site-packages (from bleach->nbconvert) (1.16.0)
Requirement already satisfied, skipping upgrade: soupsieve>1.2 in /opt/conda/lib/python3.7/site-packages (from beautifulsoup4->nbconvert) (2.3.1)
Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->nbconvert) (2.4.6)
Collecting pyzmq>=22.3
  Downloading pyzmq-22.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
     |████████████████████████████████| 1.1 MB 14.7 MB/s eta 0:00:01
Collecting python-dateutil>=2.8.2
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     |████████████████████████████████| 247 kB 31.6 MB/s eta 0:00:01
Collecting tornado>=6.0
  Downloading tornado-6.1-cp37-cp37m-manylinux2010_x86_64.whl (428 kB)
     |████████████████████████████████| 428 kB 22.4 MB/s eta 0:00:01
Requirement already satisfied, skipping upgrade: setuptools in /opt/conda/lib/python3.7/site-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (45.2.0.post20200209)
Requirement already satisfied, skipping upgrade: pyrsistent>=0.14.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (0.15.7)
Requirement already satisfied, skipping upgrade: importlib-metadata; python_version < "3.8" in /opt/conda/lib/python3.7/site-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (1.5.0)
Requirement already satisfied, skipping upgrade: attrs>=17.4.0 in /opt/conda/lib/python3.7/site-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (19.3.0)
Requirement already satisfied, skipping upgrade: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema>=2.6->nbformat>=5.1->nbconvert) (3.0.0)
Installing collected packages: traitlets, jupyter-core, pyzmq, python-dateutil, nest-asyncio, tornado, jupyter-client, fastjsonschema, nbformat, nbclient, jupyterlab-pygments, tinycss2, MarkupSafe, jinja2, nbconvert
Successfully installed MarkupSafe-2.1.1 fastjsonschema-2.15.3 jinja2-3.1.1 jupyter-client-7.2.2 jupyter-core-4.10.0 jupyterlab-pygments-0.2.2 nbclient-0.6.0 nbconvert-6.5.0 nbformat-5.3.0 nest-asyncio-1.5.5 python-dateutil-2.8.2 pyzmq-22.3.0 tinycss2-1.1.1 tornado-6.1 traitlets-5.1.1
In [9]:
!python src/create_submit_pkg.py

executing: jupyter nbconvert --to html app.ipynb
[NbConvertApp] Converting notebook app.ipynb to html
[NbConvertApp] Writing 4748578 bytes to app.html
executing: jupyter nbconvert --to html cnn_from_scratch.ipynb
[NbConvertApp] Converting notebook cnn_from_scratch.ipynb to html
[NbConvertApp] Writing 1443164 bytes to cnn_from_scratch.html
executing: jupyter nbconvert --to html transfer_learning.ipynb
[NbConvertApp] Converting notebook transfer_learning.ipynb to html
[NbConvertApp] Writing 716803 bytes to transfer_learning.html
Adding files to submission_2022-04-23T11h19m.tar.gz
src/model.py
src/data.py
src/optimization.py
src/helpers.py
src/__init__.py
src/train.py
src/create_submit_pkg.py
src/transfer.py
src/predictor.py
app.ipynb
cnn_from_scratch.ipynb
transfer_learning.ipynb
cnn_from_scratch.html
app.html
transfer_learning.html

----------------------------------------------------------------
Done. Please submit the file submission_2022-04-23T11h19m.tar.gz
----------------------------------------------------------------
In [ ]: